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Abstract — In this paper, the bit error prob- 
ability P b for maximum likelihood decoding of 
binary linear codes is investigated. The con- 
tribution of each information bit to P b is con- 
sidered. For randomly generated codes, it is 
shown that the conventional approximation at 
high SNR Ph ^ {du/N) • P,, where P s represents 
the block error probability, holds for systematic 
encoding only. Also systematic encoding pro- 
vides the minimum P b when the inverse map- 
ping corresponding to the generator matrix of 
the code is used to retrieve the information 
sequence. The bit error performances corre- 
sponding to other generator matrix forms are 
also evaluated. Although derived for codes with 
a generator matrix randomly generated, these 
results are shown to provide good approxima- 
tions for codes used in practice. Finally, for de- 
coding methods which require a generator ma- 
trix with a particular structure such as trellis 
decoding or algebraic-based soft decision decod- 
ing, equivalent schemes that reduce the bit er- 
ror probability are discussed. 

I. Introduction 

In this paper, we consider the minimization of the bit 
error probability P b for maximum likelihood decoding 
(MLD) of linear block codes. Although not optimum, 
this minimization remains important as MLD has been 
widely used in practical applications. We assume that 
the information sequence of length I\ is recovered from 
the decoded codeword based on the inverse mapping 
defined from the generator matrix of the code. For 
block codes, the large error coefficients can justify this 
strategy which is explicitly or implicitly used in many 
decoding methods such as conventional trellis decod- 
ing, multi-stage decoding or majority-logic-decoding. 
Therefore, for a particular code and the same optimal 
block error probability, we determine the best encod- 
ing method for delivering as few erroneous informa- 
tion bits as possible whenever a block is in error at 
the decoder output. We first derive a general upper 
bound on P b which applies to any generator matrix 
and is tight at medium to high signal to noise ratio 
(SNR). This bound considers the individual contribu- 
tion of each information bit separately. For randomly 
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generated codes, we then show that the systematic gen- 
erator matrix (SGM) provides the minimum bit error 
probability. To this end, a submatrix of the generator 
matrix defining an equivalent code for the bit consid- 
ered is introduced. Note that a similar general result 
holds for the optimum bit error probability related to 
the BSC [2]. We finally discuss how to achieve this per- 
formance whenever the systematic encoding is not the 
natural choice, as for trellis decoding [3] or for MLD in 
conjunction with algebraic decoding [4]-[8]. For exam- 
ple, for trellis decoding of the (32,26,4) Reed-Muller 
(RM) code, at low SNR a performance degradation of 
more than 1 dB is recovered with the proposed method. 
Minimizing the bit error probability associated with 
MLD becomes even more important whenever the con- 
sidered block code is used as the inner code of a con- 
catenated coding system [9]. 

II. Bit Error Probability for MLD 
Suppose an (N, A', dff) binary linear code C with gener- 
ator matrix G is used for error control over the AWGN 
channel. Defining 

= u) 

; = i 

where Pb(j) represents the error probability for the j th 
bit in a block of I\ information bits delivered by the 
decoder, we obtain from the union bound 

N 

PkU) < E (v'i) , (2) 
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where Q(x) = (7nV 0 ) -1 / 2 e“ n2 ^ v °dn. We call w<(j) 
the effective error coefficient associated with the 
j th information bit with respect to the genera- 
tor matrix G. 

We can prove the following theorem. 

Theorem 1 Let represent the number of codewords 
of weight i in the code C generated by G and lei uji(j) 
represent the number of codewords of weight i in the 
subcode generated by the matrix G{j) obtained after 
deleting row-j in G; then 

Wi(j) = u-i - Wi(j). (3) 

Theorem 1 depends on the mapping defined by G as 
it implicitly assumes that the inverse mapping corre- 
sponding to G is used to retrieve the information bits 



from the decoded code sequence. Since for a linear 
code, this mapping is one-to-one and thus invertible, 
Theorem 1 is valid for any representation of G, sys- 
tematic as well as non-svstematic. Combining (1) and 
(2), the average bit error probability is expressed as 
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For a code defined by a matrix G randomly generated, 
we associate with each information bit j £ [1, Tv] a 
matrix 



Da(j) = 


0 

I a - 1 
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where 1 and / a -i represent the all-1 vector and the 
identity matrix of dimension a — 1 respectively. The 
matrix D a (j) is defined as the dependency matrix 
associated with dimension j of the generator 
matrix G. This matrix allows to derive the following 
theorem. 


Theorem 2 Let consider an (//, K) linear block code 
C wtth a generator matrix generated randomly. Then 
the value Wi{j) corresponding to the dimension j with 
dependency matrix D a {j) is well approximated by 
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Theorem 2 indicates that the larger a. the larger the 
corresponding Ps(j). Consequently, a = 1 gives the 
smallest bit error probability. For this case, D\ = [1] 
which corresponds to a systematic encoding. There- 
fore, the optimum bit error probability for MLD at 
medium to high SNR is achieved by a systematic en- 
coding if the inverse-mapping defined by G is used to 
retrieve the information bits. This strategy is intu- 
itively correct since whenever a code sequence esti- 
mated by the decoder is in error, the best strategy 
to recover the information bits is simply to determine 
them independently. Otherwise, errors propagate. For 
a = 1, (6) becomes [1] 


Wi{j) =s 2 (iV » (i/N) (7) 

In that case only, at high SNR, the bit error probability 
for MLD follows 

Pb ** (if) WdH ® ( v ^") ■ ( s ) 

For Reed- Muller (RM) codes of length N < 64, we 
computed the ratios Wd H {j)/wd H corresponding to (6) 
for various forms of generator matrices. In all cases, the 
value computed from (6) is the exact ratio, although 
the weight distribution of RM codes is far from a bi- 
nomial distribution. 


III. Applications 
A. ML trellis decoding 

ML trellis decoding is based on the trellis oriented gen- 
erator matrix (TOGM) of the code considered [3]. If 
this matrix is used for encoding, trellis decoding be- 
comes suboptimum with respect to the bit error prob- 
ability of MLD. We present a simple method to over- 
come this problem. 

Let Gt denote the TOGM of the code C { . Then, by 
row additions only, it is possible to obtain the gener- 
ator matrix G of an equivalent code C which contains 
the I\ columns of the identity matrix. This matrix is 
known as the reduced echelon form (REF). These op- 
erations modify the mapping between information bits 
and codewords, but since no column permutation has 
been realized, each codeword of C is still uniquely rep- 
resented by a path in the trellis of C t . Therefore ML 
trellis decoding of the received sequence is still possible 
if we use G for encoding. The trellis decoder estimates 
the code sequence which is closest to the received se- 
quence. Then the information bits are easily retrieved 
due to the systematic nature of G. Since no restric- 
tions on Gt apply, the matrix G can be obtained for 
any possible trellis decomposition. 

In [10], a specific ML trellis decoding algorithm for 
the (63,57,3) Hamming code is proposed. The decod- 
ing is realized based on a generator matrix in cyclic 
form. It is also shown that an equivalent systematic 
representation outperforms the cyclic form by 0.4 dB 
at the BER 10~ 5 . However, the decoding of the sys- 
tematic code requires an additional step. By process- 
ing the generator matrix in cyclic form as described in 
this section, this additional step can be removed as the 
encoding matrix becomes G = [hjPe]- On the other 
hand, the cyclic structure no longer exits, but the en- 
coder remains very simple. 

Figures 1 and 2 depict the simulation results for the 
(32,16,8) and (32,26,4) RM-codes respectively. For 
both codes, we simulated ML decoding based on the 
REF and the conventional TOGM described in [3], and 
plotted the first term of the union bound derived from 
(4). As expected from the results of Section II, we ob- 
serve a larger gap in error performance for the (32,26,4) 
RM-code. At the bit error rate (BER) 10“ 6 , the gap in 
performance for this code is about 0.2 dB, which is of 
the same order as the difference between closest coset 
decoding (CCD) and ML trellis decoding [11]. Also, 
we observe a much significant gap at high BER of 0.4 
dB for the (32,16,8) code and 1.1 dB for the (32.26,4) 
code. This behavior becomes important if a concate- 
nated coding scheme is used. 

The extension of this method to multi-stage trellis 
decoding does not follow in a straightforward way. In 
general, multi-stage decoding methods exploit the de- 
composable structure of the code considered, so that 
row additions on the associated generator matrix can 



destroy this structure. For example, CCD of \u\u + v|- 
constructed codes exploits the repetition of the u- 
component code [11]. As a result, row additions in each 
component code generator matrix are allowed, but not 
from one matrix to the other. In addition, the propa- 
gation of decoding errors between decoding stages also 
has to be considered when searching for the optimum 
encoding matrix associated with multi-stage decoding. 

B. MLD in conjunction with algebraic decoding 

Several soft decision decoding algorithms in conjunc- 
tion with an algebraic decoder have been proposed [4]- 
[8]. In general, algebraic decoding is associated with a 
particular generator matrix form G a • Therefore, if this 
form is used for encoding, the corresponding algorithm 
becomes suboptimum with respect the bit error prob- 
ability of MLD. Algebraic decoding algorithms can be 
divided into two classes, depending on whether the de- 
coder delivers an estimate of the transmitted codeword 
of length N or of the information sequence of length 
I\. In the first case, the method of Section A extends 
in a straightforward fashion. Hence decoding of cyclic 
codes can be realized this way. However, a similar 
method is also possible for the second class of algebraic 
decoders. Again, this method is transparent with re- 
spect to algebraic decoding, so that the conventional 
algebraic decoder corresponding to the code consid- 
ered can still be used. This method simply consists 
of recording the row operations processed to obtain G 
in REF form G a and applying the inverse operations 
to the information sequence delivered by the algebraic 
decoder. 

Figure 3 depicts the improvement achieved by this 
method for Chase algorithm-2 with majority-logic- 
decoding for the (64,42,8) RM-code. The proposed 
method outperforms Chase algorithm-2 with conven- 
tional majority-logic-decoding by 0.15 dB at the BER 

io- 5 . 


C. Concatenated coding 

We consider the concatenated scheme presented in [12] 
where the inner code is a (64,40) subcode of the (64,42) 
RM code and the outer code is the NASA standard 
(255,223) RS code over GF( 2 s ). The outer code is in- 
terleaved to a depth of 5. For this scheme, Figure 4 
represents the simulated bit error performance for en- 
coding with the TOGM and the REF. We observe that 
the systematic encoding outperforms the TOGM by 
about 0.2 dB at the BER 10" 5 . More importantly, we 
also notice that while the error performance curves cor- 
responding to the inner codes differ by a constant value 
due to different error coefficients, the difference in bit 
error probability between the error performance curves 
corresponding to the concatenated system increases as 
the SNR increases. 


IV. Conclusion 

In this paper, we have showed that for many good 
codes, the SGM provides the best bit error probabil- 
ity for MLD when the inverse mapping of the gener- 
ator matrix G is used to retrieve the information se- 
quence. Based on the presented results, we can con- 
clude that a careful choice of the generator matrix be- 
comes important when comparing different optimum, 
near-optimum or suboptimum soft decision decoding 
schemes. Generally, tenths of dB’s separate the bit er- 
ror performance of such schemes, so that a poor choice 
of the generator matrix of one of the scheme may result 
in an important relative degradation. 

By exploiting the fact that modifying the mapping 
between information bits and codewords is transpar- 
ent to the decoder, we modified conventional trellis 
decoding and MLD in conjunction with an algebraic 
decoder so that these schemes achieve the same bit 
error performance as for systematic encoding. Hence 
the decoding becomes independent of the encoding and 
can simply be viewed as a process providing the most 
likely codeword of the codebook. As a result, the de- 
coder structure remains the same as the conventional 
one but in some cases the decoded sequence requires 
an additional simple reprocessing. 

References 

[1] F. J. Mac Williams and N. J. A. Sloane, The 
Theory of Error- Correcting Codes, Amsterdam. 
North-Holland Mathematical Library, 1977. 

[2] A. B. Kiely, J. T. Coffey and M. R. Bell, “Op- 
timal Information Bit Decoding of Linear Block 
Codes,” IEEE Transactions on Information The- 
ory , vol . IT-41, pp. 130-140, January 1995. 

[3] G. D. Forney Jr, “Coset Codes II: Binary Lattices 
and Related Codes/’ IEEE Transactions on Infor- 
mation Theory , vol. IT-34, pp. 1152-118/, Septem- 
ber 1988. 

[4] G. D. Forney Jr, “Generalized Minimum Distance 
Decoding,” * IEEE Transactions on Information 
Theory , vol. IT-12, pp. 125-131, April 1966. 

[5] D. Chase, “A Class of Algorithms for Decoding 
Block Codes with Channel Measurement Informa- 
tion,” IEEE Transactions on Information Theory , 
vol. IT-18, pp. 170-182, January 1972. 

[6] T. Kaneko, T. Nishijima, H. Inazumi and S. Hi- 
rasawa, “An Efficient Maximum Likelihood De- 
coding of Linear Block Codes with Algebraic De- 
coder,” IEEE Transactions on Information The- 
ory , vol. IT-40, pp. 320-32/. March 1994. 

[7] H. Thirumoorthv, S. Lin and T. Ivasami, Soft 
Decision Decoding of Binary Linear Block Codes 



Based on an Iterative Search Algorithm,” submit- 
ted to IEEE Transactions on Information Theory , 
(revised November 1995). 

[8] M. P. C. Fossorier and S. Lin, “Complementary 
Reliability- Based Decodings of Binary Linear Block 
Codes,” submitted IEEE Transactions on Informa- 
tion Theory , December 1995. 

[9] G. D. Forney Jr. Concatenated Codes' \ Cam- 
bridge: M.I.T. Press, 1966. 

[10] A. M. Michelson and D. F. Freeman, “Viterbi De- 
coding of the (63,57) Hamming Codes - Implemen- 
tation and Performance Results,” IEEE Transac- 
tions on Communications, vol. COM-43, pp. 2653- 
2656, November 1995. 

[11] F. Hemmati, “Closest Coset Decoding of |u|u-ft>| 
Codes,” IEEE Journal on Selected Areas in Com- 
munications , vol. JSAC-7, pp. 982-988, August 
1989. 

[12] T. Kasami, T. Takata, K. Yamashita, T. Fujiwara 
and S. Lin. “On the Bit Error Probability of a Con- 
catenated Coding Scheme,” The Proceedings Inter- 
national Symposium on Information Theory and its 
Applications , Sydney, Australia, p. 1-6, November 
1994, submitted IEEE Transactions on Communi- 
cations \ 1995. 



Figure 1: Simulated and theoretical bit error probabil- 
ities for the (32,16,8) RM code with TOGM and REF. 



Figure 2: Simulated and theoretical bit error probabil- 
ities for the (32,26,4) RM code with TOGM and REF. 



Figure 3: Simulated bit error probabilities for Chase 
algorithm-2 of the (64,42,8) RM code with majority- 
logic-decoding in Boolean and systematic forms. 



(255,223) RS outer code and (64,40,8) inner code, and 
encoding with REF and TOGM. 





